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Abstract —  Networks  of  different  types  are  essential 
infrastructures  for  society.  Networks  such  as  transportation 
networks  and  the  electric  power  grid  are  becoming  increasingly 
large  and  complex  which  create  special  challenges  for  human 
operators  who  must  monitor  these  networks  for  safe  operation.  It 
is  important  to  understand  how  human  operators  monitor  these 
networks  and  the  challenges  they  face  in  the  monitoring  task.  In 
order  to  do  this,  we  conducted  a  survey  of  how  human  operators 
monitor  various  networks — water  distribution  networks,  electric 
power  grid,  air  traffic  control,  and  nuclear  power  plant 
networks.  From  our  survey  in  these  specific  extensively-studied 
networks,  we  are  able  to  generalize  the  challenges  of  operators 
monitoring  networks.  The  hope  is  that  these  results  will  provide 
human  factors  engineers  with  a  better  understanding  so  they  can 
cater  to  the  needs  of  the  human  operators  when  designing  new 
interfaces  for  network  monitoring  tasks. 

Keywords:  network  monitoring,  human  factors,  visualization 

1.  INTRODUCTION 

Networks  that  enhance  our  quality  of  life  are  increasingly 
becoming  more  complex  in  size,  speed,  and  functionality  with 
the  advent  of  technological  advances.  Because  these  networks 
are  interdependent,  all  these  infrastructures  are  built  upon 
underlying  computerized  systems  that  are  interconnected;  they 
cannot  be  studied  completely  in  isolation. 

These  dimensions  of  network  complexity  have  many 
implications.  In  this  paper  we  focus  on  the  problem  of  human 
monitoring  of  increasingly  complex  networks.  Human 
monitored  networks  are  prone  to  adverse  consequences 
because  human  cognitive  abilities  to  perceive,  understand, 
decide,  and  react  do  not  scale  in  the  same  way  as  network 
size,  speed,  and  functionality. 

How  well  networks  are  monitored  by  human  operators  not 
only  has  a  great  impact  on  the  productivity  of  these  networks 
but  also  on  safety — in  fact  eight  of  these  networks  have  been 
labeled  critical  infrastructures  for  special  protection1.  Because 

1  Presidential  Executive  Order  on  Critical  Infrastructure  Protection  13010, 
Federal  Register,  July  17,  1996,  Vol  61  No  138,  pp.  37347-37350.  The  eight 
critical  infrastructures  listed  are:  (1)  Telecommunications,  (2)  Electric  Power 
Systems,  (3)  Gas/Oil  Storage/Transportation,  (4)  Banking  and  Finance,  (5) 
Transportation  (surface  and  aviation),  (6)  Water  Supply  System,  (7) 
Emergency  Services  (Fire  &  Rescue,  Law  Enforcement,  Public  Health),  and 
(8)  Continuity  of  Government. 


of  this,  it  is  important  to  understand  how  human  operators 
monitor  these  networks — the  tasks  that  they  perform,  the 
challenges  they  face,  the  current  tools  that  they  use.  Using 
this  information,  designers  can  then  create  interfaces  that 
enhance  the  human  operation  to  be  more  successful  at 
monitoring  the  network — alleviating  information  overload, 
integrating  information,  highlighting  information,  decreasing 
response  time  and  increasing  effectiveness. 

In  order  to  understand  how  human  operators  monitor  these 
complex  networks,  we  conducted  a  literature  survey  that 
examined  various  complex  networks  and  the  tasks  involved  in 
monitoring  the  network,  the  current  tools  used,  and  the 
challenges  operators  face.  As  a  result  of  this  survey,  we  have 
identified  common  challenges  and  addressed  design 
implications  for  monitoring  such  networks. 

The  remainder  of  this  paper  is  organized  as  follows:  Section 
II  defines  terminology  and  the  scope  we  will  be  investigating. 
Section  III  is  a  detailed  literature  survey  of  four  different 
networks.  Section  IV  discusses  insights  gained  from  the 
survey  information  in  terms  of  common  and  contrasting 
issues.  We  end  with  a  summary  and  conclusions  in  Section  V. 

II.  DIFFERENT  TYPES  OF  NETWORKS 

For  the  purposes  of  this  paper,  we  define  a  network  as  an 
interconnecting  collection  of  processing  nodes  and 
transmission  links  that  change  dynamically  over  time  to  move 
some  entity  (i.e.  electricity)  from  one  point  to  another  as 
efficiently  as  possible  within  a  budget  constraint.2  A  network 
can  be  represented  with  two  types  of  information:  (1)  the 
topology  or  structure  of  interconnecting  nodes  and  links  and 
(2)  associated  weights  on  each  node  and  link  corresponding  to 
costs,  capacity,  supply  demands,  etc.  The  general  area  of 
networks  can  be  categorized  into  smaller  and  more  specific 
type  of  networks.  For  example,  in  our  study  of  human 
monitoring  of  networks,  the  following  types  of  networks 
arose — process  intensive  and  node  intensive.  A  process 
intensive  network  requires  a  relatively  high  amount  of 
manipulation  and  processing  of  data,  whether  at  a  given  node 
or  processing  across  edges,  such  as  the  air  traffic  control 
network.  The  information  about  each  individual  aircraft  needs 

2  Adapted  from  Network  Flows:  Theory,  Algorithms,  and  Applications  by 
R.  Ahuja,  R.L.  Magnanti,  J.B.  Orlin,  Prentice  Hall,  1993. 


to  be  processed,  integrated,  and  constantly  updated  in  the 
system  to  give  air  traffic  controllers  the  overall  picture  of  a 
specified  area  of  air  traffic  in  real-time;  thus,  there  is  an 
intensive  processing  requirement  at  specific  nodes.  Similarly, 
a  node  intensive  network  is  a  network  in  which  there  are  a 
large  number  of  nodes  in  a  network.  For  example,  a  nuclear 
power  plant  can  be  viewed  as  a  collapsed  network  of  nodes 
that  provide  measurements  and  data  values  to  operators  about 
the  state  of  the  plant. 

Much  work  has  been  conducted  to  aid  human  operators 
improve  productivity,  reduce  response  times,  and  be  more 
vigilant  for  each  of  the  networks  we  will  examine.  However, 
there  has  been  little,  if  any,  work  seeking  to  find  common 
tasks  between  monitoring  of  different  types  of  networks. 

III.  HUMAN  MONITORING  SURVEY 

We  chose  to  conduct  a  literature  survey  in  the  areas  of 
water  distribution  systems,  electric  power  grids,  air  traffic 
control,  and  nuclear  power  plants.  Each  of  these  networks  has 
been  extensively  studied  and  we  draw  on  this  research.  For 
each  network  we  identify:  (1)  the  monitoring  task  that  a 
human  operator  performs,  (2)  the  current  tools  that  are  being 
used  by  the  operators,  (3)  the  challenges  that  the  operators 
face  in  performing  their  monitoring  task,  and  (4)  proposed 
solutions  to  aid  the  human  operator.  We  present  the  networks 
in  approximate  historical  order  to  emphasize  the  evolution  of 
human  network  monitoring  in  different  contexts. 

A.  Water  Distribution  Systems 

Monitoring  Task(s) 

The  day-to-day  operation  of  a  water  system  network  is 
typically  the  responsibility  of  one  to  two  plant  operators,  in 
addition  to  assistants,  who  are  on  duty  at  all  times  at  the 
treatment  plant.  Their  main  task  is  to  monitor  the  quality  of 
water  as  it  enters  the  distribution  system.  Typically,  this 
monitoring  is  not  performed  in  real-time  except  for 
monitoring  pressure  levels  (i.e.  to  catch  water  main  breaks) 
and  sometimes  chlorine  residual  levels  [1]. 

In  addition  to  monitoring  water  plants,  the  water  system 
needs  to  deliver  clean  water  and,  conversely,  prevent  drinking 
water  contamination  [2].  In  order  to  prevent  contamination, 
the  United  States  is  now  moving  towards  the  use  of  real-time 
monitoring  of  water  distribution  sites  due  to  security  concerns 
as  well  as  improvements  in  real-time  water  monitoring 
equipment.  Operators  need  to  reliably  identify  low 
probability/high  impact  contamination  events  in  source  water 
or  distribution  systems  in  order  for  a  timely  and  effective 
local  response  that  avoids  or  mitigates  adverse  impacts  [2]. 

Current  Tools 

The  operational  monitoring  and  control  of  water  systems  is 
handled  with  Supervisory  Control  and  Data  Acquisition 
(SCADA)  systems.  SCADA  can  have  a  graphical  interface 
component  that  shows  a  general  system  overview  as  well  as 
specific  details  such  as  reservoir  and  water  tower  levels,  status 


Figure  1.  Water  Plant  Control  Room 

<http://www.oswego.edu/prosmart/PROJECTS.HTM> 

of  pump  stations,  filters,  tanks,  pressure  gauges  and  other 
components  linked  by  pipes. 

Because  monitoring  of  drinking  water  for  contamination  is 
not  required,  there  are  currently  no  tools  being  used  in  water 
distribution  plants  for  real-time  monitoring  of  the  water 
drinking  quality;  however,  much  research  is  being  conducted 
in  this  area. 

Challenges 

There  are  many  challenges  to  real-time  water  system  to 
monitoring.  One  big  challenge  is  determining  the  number  of 
monitoring  stations  needed  to  accurately  detect  the 
contamination  of  the  drinking  water.  Though  the  likelihood 
of  detecting  contamination  increases  with  more  monitoring 
stations,  there  is  a  tradeoff  of  diminishing  returns — the 
amount  of  detection  gain  diminishes  with  each  new 
monitoring  stations  [3].  In  addition,  as  the  number  of 
monitoring  stations  increases  to  a  certain  level,  the 
information  provided  will  often  be  redundant  increasing  both 
information  overload  and  decreasing  the  percentage  of 
relevant  information  [3]. 

Another  challenge  for  the  monitoring  of  a  water  system  is 
the  need  for  operators  to  verify  what  is  happening  when  a 
contamination  occurs  against  what  is  predicted  in  a 
computational  model  [2].  Not  only  do  operators  need  to 
identify  and  verify  contamination,  but  they  also  need  to 
identify  the  point  of  intrusion  from  information  provided  by 
monitoring  stations  [4]. 

Proposed  Solutions 

Computer  modeling  can  be  used  to  find  the  optimal  layout 
and  number  of  monitoring  stations  for  a  water  system  which 
can  reduce  the  cognitive  demands  of  operators  [4].  An 
optimal  infrastructure  design  will  more  effectively  track  and 
potentially  contain  contamination  throughout  a  water 
distribution  system  [4]. 

To  aid  operators  in  the  task  of  real-time  monitoring  of  a 
water  network,  a  system  was  created  that  automatically 


validates  information  from  mathematical  hydraulic  and  water 
quality  model  with  that  of  the  data  gathered  by  the  SCADA 
system,  MIKE-NET-SCADA  [2],  MIKE-NET-SCADA  takes 
data  from  the  SCADA  system  to  validate  against  a  model. 
With  an  integrated  modeling  approach,  the  monitoring  of 
hydraulic  conditions  in  the  water  distribution  network  can  be 
automated  allowing  an  operator  to  concentrate  on  monitoring 
entire  water  network  operation  instead  of  focusing  on  local 
data  from  a  limited  number  of  SCADA  sensors  [2], 
Additional  benefits  include  an  enhanced  ability  to  process 
larger  amounts  of  information  in  a  shorter  time  and  emergency 
responsiveness. 

B.  Electric  Power  Grids 
Monitoring  Task(s) 

Power  grid  operators  have  many  tasks  in  their  day-to-day 
monitoring  of  the  grid.  The  three  main  monitoring  tasks  are: 
(1)  monitoring  sensor  data  from  power  grid  components  (bus 
voltages,  bus  numbers,  status  of  the  capacitors,  transmission 
lines,  transformers,  circuit  breakers,  etc.),  (2)  monitoring  the 
grid  as  a  holistic  system,  and  (3)  responding  to  alarms  [5]. 

Current  Tools 

To  obtain  an  overview  of  the  status  of  the  grid,  operators 
use  a  map  board.  The  map  board  is  static  map  of  the  overall 
system,  but  the  data  is  dynamic  and  displayed  using  different 
colored  lights  [6].  In  addition,  the  map  board  also  displays  the 
status  and  alarm  conditions  of  individual  transmission 
elements  [7].  Because  the  map  board  only  provides  an 
overview  of  the  grid,  the  map  board  is  then  supplemented  by 
computer  displays  that  show  system  measurements  using  a 
combination  of  data  tables  and  vector-based  graphics  [7]. 

Traditionally,  the  computer  displays  that  provide  a  more 
detailed  picture  of  the  plant  use  tabular  list  displays,  one-line 
diagrams,  or  contour  diagrams  to  display  data  [5].  A  tabular 
list  is  a  list  of  tables  that  display  various  pieces  of  information 
about  the  grid — bus  voltage  magnitudes,  status  of  the 
capacitors,  and  line  outages.  All  the  information  that  is 
presented  by  a  tabular  list  is  text-based,  but  the  text  may  be 
displayed  in  another  color  (i.e.  red)  if  it  exceeds  or  falls  below 
a  certain  threshold  [6].  One-line  diagrams  are  diagrams  of  the 
system  showing  the  buses  and  the  bus  voltage  magnitude 
associated  with  the  bus.  If  the  bus  voltage  falls  beneath  a 
certain  voltage,  the  bus  becomes  red  [6].  Contouring 
diagrams  are  the  same  as  one-line  diagrams,  except  the  bus 
lines  are  colored  coded  based  upon  the  voltage  and  the  color 
contour  associated  with  that  particular  number  [6]. 

Challenges 

Because  a  power  grid  contains  so  many  different  parts  and 
connections  between  the  parts,  the  monitoring  task  can  be 
especially  challenging  for  an  operator.  The  operator  not  only 
needs  to  monitor  the  equipment  and  status  of  the  equipment, 
but  they  also  need  to  understand  how  the  equipment  flows  and 
how  they  are  interconnected  [7,  8].  Because  of  this  need  to 
see  an  overview  and  also  the  details  of  the  grid,  one  big 


Figure  2.  Electric  Power  Grid  Control  Room 

<http://msnbc.msn.com/id/3077316/> 

challenge  is  integrating  an  overview  and  detailed  context  of 
the  grid  to  give  an  operator  the  maximal  information  without 
overload.  This  requires  a  balance  of  understanding  what 
information  is  pertinent  to  display  in  the  overview  and  the 
detailed  context  for  the  operator  and  what  information  would 
cause  information  overload.  Currently,  the  overview  of  the 
plant  is  provided  through  the  board  map  that  contains  little  or 
no  numerical  information,  and  the  detailed  context  is  located 
on  separate  displays  for  individual  substations  [7].  Even  with 
the  overview  and  context  displays,  it  is  difficult  and  unlikely 
that  all  data  can  be  presented  to  an  operator  due  to  the  lack  of 
real  estate  on  various  displays  [9]. 

Another  challenge  that  arises  is  how  the  information  is 
presented  to  the  operator.  The  first  part  of  data  presentation  is 
to  determine  the  layout.  Many  plants  use  a  data  display  driven 
primarily  by  the  structure  of  the  power  system.  Specifically, 
the  data  appears  grouped  by  geographic  proximity  in  the 
power  system  [9].  Using  this  display  structure  has  a  tradeoff 
between  context  and  task.  The  use  of  this  layout  provides  an 
operator  context  of  the  data  and  the  ability  to  understand  the 
relationship  between  pieces  of  data.  Unfortunately  this  layout 
can  reduce  the  efficiency  and  productivity  of  an  operator 
because  this  layout  can  require  an  operator  to  look  at  data 
scattered  throughout  the  system  to  perform  a  particular  task. 

Another  challenge  is  how  to  effectively  display  the  data  to 
the  operator  so  that  they  can  perform  the  monitoring  tasks 
effectively  and  efficiently.  Data  can  be  displayed  by  the  use 
of  numbers,  colors,  diagrams,  or  any  combination  of  the  three. 
Depending  on  the  needs  of  the  operator,  certain  display 
presentations  maybe  better  than  others.  For  example, 
displaying  data  solely  using  numbers  allows  an  operator  to 
obtain  exact  information,  but  it  requires  many  steps  for  the 
operator  to  assess  the  state  of  the  plant — reading  the  value, 
consulting  a  table  of  values  to  retrieve  the  limits  that  define 
the  state  for  the  specific  situation,  comparing  the  values 
received  from  the  display  with  that  in  the  table,  and  finally 
determining  the  status  of  the  value  [8].  The  use  of  numbers 


also  makes  it  difficult  for  an  operator  to  identify  trends  in  the 
system  to  assess  the  system’s  behavior.  Moreover,  displaying 
data  based  solely  on  color  may  allow  operators  to  assess  the 
state  of  the  system  more  quickly.  There  is  a  disadvantage  to 
using  only  color  in  the  displays  because  it  can  be  more 
difficult  and  time  consuming  to  resolve  any  problems  that 
arise  because  of  the  lack  of  specific  data,  in  addition,  data  can 
be  presented  through  diagrams  of  the  system  which  is 
generally  coupled  with  numerical  data.  This  allows  for  an 
operator  to  identify  specific  parts  of  the  system  that  needs 
immediate  attention  when  a  problem  arises  as  well  as  the 
specific  data  values. 

Proposed  Solutions 

The  challenges  that  arise  from  monitoring  a  power  grid  are 
related  to  retrieving  and  presenting  the  data  from  the  system. 
In  order  to  alleviate  the  overwhelming  amounts  of  information 
for  operators,  research  has  been  conducted  on  integrating  the 
data  into  different  forms  for  operators,  in  particular,  research 
has  been  conducted  in  the  area  of  using  diagrams  and 
graphical  presentation  [6].  Research  has  shown  that  with 
visual  presentation,  users  are  able  to  not  only  detect  problems 
in  the  system,  but  are  better  able  to  identify  and  execute  the 
appropriate  solution  [6].  The  reason  for  this  is  that  human 
ability  to  recognize  patterns  and  connections  in  graphical 
presentations  significantly  reduces  the  amount  of  cognitive 
effort  required  to  make  decisions  [9].  Despite  this  visual 
advantage,  interface  designers  for  power  plant  grids  still  need 
to  determine  what  information  should  be  presented  in  the 
diagrams  and  how  they  should  be  presented. 

C.  Air  Traffic  Control 
Monitoring  Task(s) 

There  are  three  different  types  of  air  traffic  control  facilities: 
(1)  the  air  traffic  control  tower,  (2)  the  terminal  radar 
approach  control  (TRACON),  and  (3)  the  air  route  traffic 
control  center  (en  route  center)  [10]. 

The  main  task  of  the  air  traffic  control  center  is  to  control 
aircraft  on  the  ground  at  two  critical  points — just  before 
takeoff  and  just  before  landing.  The  tasks  are  usually  shared 
with  the  ground  controller  and  the  local  area  controller. 
Specifically  the  air  traffic  control  towers  tasks  include  (1) 
clearing  aircraft  to  push  back  form  the  gate,  (2)  clearing 
aircraft  to  leave  the  ground,  (3)  managing  ground  traffic  to 
and  from  the  gate  and  runways,  (4)  maintaining  a  safe 
distance  separation  between  aircraft,  and  (5)  handoff  of 
aircraft  with  the  TRACON  controller  [10]. 

The  tasks  of  the  TRACON  operators  are:  (1)  to  manage  safe 
and  expeditious  flow  of  departing  aircraft  accepted  from  the 
tower  to  a  handoff  to  the  en  route  controller,  (2)  manage  the 
arriving  aircraft  from  a  handoff  from  the  en  route  controller  to 
a  handoff  to  the  tower  controller  on  a  final  approach  for 
landing,  (3)  sequencing  the  aircraft  in  an  orderly  inbound  or 
outbound  flow  at  a  regular  spacing,  and  (5)  maintaining  a  safe 
distance  separation  between  aircraft  [10]. 

Operators  in  the  en  route  center  need  to  use  the  radar 
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information  to  provide  guidance  to  flying  aircraft.  They  also 
need  to  maintain  the  expeditious  delivery  of  an  aircraft  stream 
to  the  receiving  TRACONs  [10]. 

All  air  traffic  control  facilities  share  the  responsibility  of 
monitoring  individual  aircraft  movement  with  surrounding 
aircraft.  If  any  problems  arise,  any  of  the  three  air  traffic 
control  facilities  may  alert  aircraft  pilots  to  initiate  changes 
from  filed  plans  [11]. 


Current  Tools 

With  each  facility,  operators  may  rely  on  different  tools  in 
order  to  complete  their  tasks.  Because  the  air  traffic  control 
tower  has  the  most  complex  tasks  in  monitoring  aircraft, 
operators  rely  on  a  variety  of  tools:  their  eyes, 

communications  link,  airport  surface  detection  equipment 
(ASDE),  DBRITE,  and  paper  flight  strips.  Because  the  main 
tasks  of  operators  in  the  tower  are  to  manage  the  aircraft  on 
the  ground  as  they  move  from  taking  off,  landing,  and  taxiing 
on  the  ground,  operators  use  their  eyes  to  monitor  where  the 
location  of  aircraft  are  [10].  Operators  also  use  voice 
communication  link  to  verify  the  location  of  aircraft.  When 
there  is  reduced  visibility,  either  from  weather  or  nighttime, 
operators  face  challenges.  Fortunately,  the  airport  surface 
detection  equipment  (ASDE)  allows  operators  to  monitor 
aircraft  on  the  ground  even  when  visual  contact  cannot  be 
made.  The  ASDE  is  a  system  that  provides  radar 
identification  of  ground  vehicles  and  aircraft  in  an  airport. 
Operators  also  use  DBRITE  which  is  a  radar  display  that 
augments  visual  control  of  the  airborne  traffic.  DBRITE 
provides  the  operator  with  a  radar  presentation  of  about  fifteen 
miles  around  the  airport,  alphanumeric  information  on  the 
aircrafts  that  are  received  from  the  automated  radar  terminal 


system  nearby  TRACON,  view  of  arrival  aircraft  that  are  not 
under  the  tower’s  control  so  that  the  aircraft  that  are  under  the 
tower’s  responsibility  on  the  ground  can  be  safely  and 
efficiently  coordinated  with  planned  arrivals.  In  addition  to 
these  tools,  operators  also  use  paper  flight  strips,  which  are 
physical  representations  of  each  aircraft,  as  a  visible  reminder 
of  an  aircraft’s  status  in  the  sequence  of  taxi-takeoff. 

TRACON  operators  use  a  radar  monitoring  tool,  automated 
radar  terminal  system  (ARTS)  that  provides  information  about 
departing/landing  aircraft  providing  operators  a  big  picture  of 
aircraft  movements  in  his/her  assigned  area.  ARTS  also 
provide  TRACON  operators  with  conflict  alerts,  minimum 
safe  altitude  warnings  and  predict-and-confirm  system  for 
aircraft  movements.  Similar  to  operators  in  towers  and 
TRACONS,  en  route  operators  also  use  radar  to  monitor  air 
traffic. 

Challenges 

Many  of  the  challenges  that  arise  from  monitoring  aircraft 
arise  from  cognitive  vulnerabilities  while  on-task.  In  visual 
sampling  and  selective  attention,  controllers  are  vulnerable  to 
missing  critical  events  through  breakdowns  in  the  serial 
scanning  process.  When  scanning  radars,  controllers  face 
display  clutter,  aircraft  misidentification,  and  may  not  be  able 
to  detect  unexpected  events  due  to  expectations. 

Working  memory  is  another  area  of  vulnerability  for 
operators.  Working  memory  is  susceptible  to  interference 
from  competing  processes  (i.e.  speaking  will  disrupt  verbal 
working  memory;  visual  scanning  and  search  will  disrupt 
spatial  working  memory).  Working  memory  also  suffers 
when  it  must  retain  items  that  are  similar  to  each  other,  such 
as  aircraft  with  similar  call  signs.  Most  situational  awareness 
is  heavily  dependent  on  spatial  working  memory  to  compute 
future  states  of  aircraft.  Calculating  the  future  states  of  the 
aircraft  can  be  very  taxing  on  controllers  due  to  the 
complexity  of  calculating  trajectories  from  radars,  crowding  in 
the  airspace,  and  potential  loss  of  radar  data.  In  addition, 
other  areas  of  cognitive  vulnerabilities  include 
communications,  long-term  memory,  judgment  and  decision 
making,  and  errors  [10]. 

The  displays  used  by  controllers  can  pose  as  a  potential 
problem  area  for  controllers.  One  area  in  particular  is  the 
ability  to  detect  traffic  conflicts  [11].  During  conflict 
detection,  controllers  rely  on  the  visual  display  of  information 
to  extrapolate  of  aircraft  pair  trajectories  [11].  This  task  is 
made  even  more  difficult  if  the  display  overloads  the 
controller  with  too  many  aircraft.  Furthermore,  these  displays 
depend  on  alphanumeric  data  blocks  for  critical  information. 
Because  of  this,  numerous  eye  fixations  and  complex  mental 
transformations  are  required  for  a  controller  to  form  a  3D 
mental  representation  of  the  location  and  heading  of  an 
aircraft  [12]. 

Proposed  Solutions 

Air  traffic  monitoring  is  a  very  complex  task  and  taxing 
upon  a  controller.  One  way  that  researchers  have  been 
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looking  at  reducing  this  workload  is  changing  the  way  in 
which  information  is  displayed.  Research  has  suggested  the 
use  of  color  coding  altitude  to  reduce  the  time  to  detect 
conflicts  in  static  air  traffic  control  [11].  By  doing  this, 
controllers  get  pre-attentive  situational  awareness.  Research 
is  needed  to  improve  information  displays  and  decrease 
cognitive  vulnerabilities. 

D.  Nuclear  Power  Plants 
Monitoring  Task(s) 

Nuclear  power  plants  have  thousands  of  parameters  that 
operators  can  potentially  monitor — the  control  room  of  a 
nuclear  power  plant  typically  has  hundreds  of  indicators.  The 
task  of  monitoring  the  plant  does  not  merely  involve  reading 
indicators  and  values  to  see  if  they  fall  in  a  safe  threshold. 
Operators  of  nuclear  power  plants  need  to  decide  what  data 
and  information  is  relevant  to  monitor  and  what  deviation  or 
change  in  these  values  is  meaningful  [13].  The  monitoring 
task  is  dependent  upon  the  prior  knowledge  of  operators  such 
as  the  unique  properties  of  plant  systems  and  how  systems 
interact  [14]. 

Typically  the  monitoring  task  requires  operators  to  answer 
higher  order  questions  from  a  single-sensor,  single-indicator 
displays.  To  answer  these  higher  order  questions  (i.e.  the 
current  state  of  the  plant),  operators  need  to  determine  which 
pieces  of  data  are  relevant  for  the  current  task,  gather  the 
relevant  data  from  individual  instruments  (which  are  often 
located  on  different  control  boards),  and  integrate  this 
information  mentally  to  derive  the  higher-order  properties  of 
interests  [14,  15]. 

Current  Tools 

In  [13],  a  thorough  case  study  was  performed  to  see  how 
nuclear  power  plant  operators  monitor  the  plant  and  revealed 
the  tools  that  operators  use  for  their  tasks.  The  primary  source 
of  information  for  monitoring  comes  from  the  control  room 
indicators  and  alarms.  The  control  room  panels  and  CRT 
displays  were  found  to  be  monitored  on  a  regular  basis  by 


virtually  all  the  operators  observed.  The  CRTs  are  dedicated 
to  presenting  textual  alarm  messages  as  a  chronologically 
ordered  list.  Newer  control  rooms  use  the  CRTs  to  display 
graphs  and  trends  [15].  With  the  alarm  displays,  there  is  an 
auditory  signal  accompanied  by  the  onset  and  offset  of  an 
alarm  in  addition  to  messages  displayed  on  the  CRT.  Because 
of  the  saliency  of  the  alarm,  they  are  a  frequent  source  of 
information  for  monitoring.  Operators  can  also  gain 
information  from  communication  and  logs  about  the  plant  to 
aid  in  their  monitoring  of  the  plant.  Operators  tended  to 
communicate  with  other  operators  during  shift  turnovers  in 
order  to  understand  the  status  of  the  plant  and  any  situations 
that  they  need  to  be  aware  of.  Operators  in  the  control  room 
also  communicated  with  other  operators  located  in  the  plant 
during  their  shift  to  obtain  information  about  certain  systems 
and  equipment.  Historical  information  documented  in  logs 
and  longer-term  status  binders  tend  to  be  used  as  reminders  to 
the  operators  of  what  has  been  done  in  the  previous  shifts  or 
in  the  past  several  weeks.  In  addition,  operators  also  utilized 
plant  walk-throughs  as  a  means  of  providing  a  background,  or 
context,  for  monitoring  [13]. 

Challenges 

Because  of  the  age  of  some  nuclear  power  plants,  the 
systems  that  are  used  for  monitoring  can  be  unreliable.  In 
[13],  researchers  found  there  was  always  components, 
instruments,  or  subsystems  that  are  missing,  broken,  working 
imperfectly,  or  being  worked  on.  This  changes  the  way  that 
the  operators  interpret  the  information  from  the  system 
because  they  have  difficulty  establishing  what  is  normal  and 
correct  due  to  unreliable  sensors.  This  is  further  exacerbated 
by  the  fact  that  systems  are  complex  and  interrelated,  which 
makes  it  difficult  to  understand  the  implications  of  failures. 

The  alarm  system  also  poses  a  challenge.  The  alarm  system 
produces  alarms  that  are  not  context  sensitive  leading  to  many 
nuisance  alarms.  For  instance,  the  same  alarm  may  appear  if  a 
certain  component  is  being  repaired  or  if  a  certain  component 
is  not  working  correctly  (Byzantine  errors).  Due  to  many 
false  positives,  operators  may  spend  an  inordinate  amount  of 
time  filtering  alarms  [13]. 

The  display  design  is  another  challenge.  Operators  must 
monitor  a  display  of  instmments  serially  remembering 
positions  and  values.  Typical  displays  are  cluttered  and 
difficult  to  read  because  the  instmments  are  so  closely  packed 
and  the  indicator  and  its  proximity  to  the  set  point  are  difficult 
to  read  from  a  distance  [13].  It  is  difficult  to  derive  higher- 
order  status  from  low  level  instrument  data  such  that  diagnosis 
of  the  nature  of  abnormalities  is  problematic  [15]. 

Stress  can  have  an  adverse  effect  on  operator  performance. 
The  stress  placed  on  an  operator  could  be  the  result  of 
workload,  competing  goals,  uncertain  conditions,  and  other 
factors  that  can  contribute  to  a  loss  of  control.  During  high 
stress  situations,  performance  on  tasks  that  require  integration 
of  many  cues  or  decision  making  that  requires  consideration 


of  many  options  may  be  impaired  because  of  decreased  ability 
to  allocate  attention  to  peripheral  cues.  Furthermore,  stress 
can  result  in  a  failure  of  work  teams  to  pool  information, 
thereby  jeopardizing  effective  situational  assessment  [16]. 

Proposed  Solutions 

In  the  case  study  presented  in  [13],  researchers  found  that 
operators  were  better  able  to  perform  their  duties  by  adapting 
with  new  strategies  to  help  their  monitoring  task.  For 
instance,  operators  maximized  the  information  that  could  be 
extracted  from  data  through  experience  with  the  system. 
Operators  also  used  the  equipment  to  create  new  information 
by  creating  a  new  indicator  or  alarm.  Moreover,  operators 
could  create  external  cues  as  reminders  [13]. 

Information  can  be  better  presented  to  operators.  One  area 
of  research  has  focused  on  how  to  integrate  low-level  data 
into  high-level  information  that  an  operator  can  more  easily 
process  and  understand  [14].  In  this  way,  operators  can 
monitor  for  functional  goals  rather  than  sensor  level  values 
[14].  This  integration  can  also  reduce  the  amount  of  data  that 
an  operator  needs  to  monitor  by  having  the  system  automate 
the  task  of  collecting  and  structuring  the  data  for  the  operator 
[14].  This  is  not  to  say  that  displays  in  the  control  room  be 
eliminated,  but  that  there  should  be  a  display  that  allows 
operators  to  see  the  status  of  the  plant  on  a  high-level  through 
the  integration  of  data.  If  there  is  a  problem,  the  operators  can 
then  look  at  the  specific  values  which  might  be  the  cause  of 
the  abnormalities  or  errors. 

With  the  alarm  system  display,  the  alarms  can  be  made  to 
be  more  context  sensitive.  The  system  might  be  designed 
such  that  operators  are  allowed  to  customize  what  messages 
are  displayed  so  that  the  number  of  nuisance  alarms  is 
minimized;  thus,  the  amount  of  filtering  that  an  operator  needs 
to  perform  is  reduced. 

IV.  ANALYSIS  OF  NETWORK  MONITORING  TASKS 

The  networks  we  have  studied  are  monitored  by  human 
operators  because  humans  are  able  to  creatively  handle  new 
situations  that  will  inevitably  arise.  In  order  to  be  effective  in 
this  way,  human  operators  require  a  situational  understanding 
of  the  network  which  is  increasingly  difficult  as  systems 
become  more  opaque,  more  complex,  and  interdependent  [17]. 
While  most  network  monitoring  tasks  can  be  characterized  as 
normal  day-to-day  activities,  when  problems  do  arise 
operators  then  need  to  perform  some  combination  of  the 
following  tasks:  cue  interpretation,  hypothesis  generation, 
hypothesis  selection,  and  action  selection  [18].  We  found  in 
the  networks  we  studied  that  using  visualization  for  network 
monitoring  is  the  primary  means  for  supporting  humans  with 
these  requirements  by  providing  situational  awareness,  quick 
gathering  of  information  [18],  and  a  means  to  convey 
uncertainty  to  the  decision-maker  [19]. 


A.  Challenges  of  monitoring 

In  conducting  the  survey,  we  discovered  common  themes  in 
the  challenges  that  operators  face  when  monitoring  a  complex 
network: 

•  Experiencing  data  overload.  We  found  that  operators 
can  be  overwhelmed  with  the  amount  of  information 
that  they  need  to  monitor  [5-8,  10,  13].  Visualization 
was  the  most  effective  technique  for  representing  data 
in  order  to  reduce  this  overload. 

•  Difficulty  in  filtering  data.  With  the  large  amounts  of 
data  available  to  operators,  operators  can  potentially 
spend  a  majority  of  their  time  sifting  through  data  in 
order  to  find  data  that  will  add  to  their  knowledge  of 
the  state  of  the  network  [13]. 

•  Querying  for  information  about  the  network.  Operators 
can  have  difficulty  in  obtaining  the  data  that  they  need 
in  the  form  that  they  need.  For  example,  operators  may 
need  an  overview  of  the  state  of  the  network,  but  only 
low-level  information  (i.e.  sensor  data)  is  available; 
thus,  operators  may  have  difficulty  in  assessing  the 
current  state  of  the  network  [2]. 

•  Human  operators  respond  to  network  management 
tasks  along  a  continuum  between  proactive  and 
reactive.  In  proactive  monitoring,  operators  seek  to 
determine  the  current  system  state  and  then  predict  the 
trajectory  of  the  future  state  of  the  network.  This  is 
most  prominently  seen  with  air  traffic  controllers  that 
need  to  predict  the  trajectory  of  airplanes  to  prevent 
congestion  or  collisions  [10].  In  reactive  monitoring, 
operators  do  not  attempt  to  assess  the  future  state  of  the 
network,  but  rather  only  react  to  alarms  that  explicitly 
define  specific  problems  and  then  follow  an  official 
protocol  to  address  these  problems  [6]. 

Table  I  represents  these  challenges  based  upon  the  type  of 
monitoring  performed  by  the  operator  as  well  as  by  common 
attributes  of  the  monitoring  task-  data  visualization  and 
interactive  query  capability  for  information  about  the  network. 

We  found  that  information  about  the  network  is  presented 
in  various  formats  to  the  human  operators  (visual,  numerical, 
verbal,  tactile)  in  the  different  networks  we  studied.  Research 
results  confirm  our  intuition  that  visual  communication  is 
more  effective  than  either  numerical  or  verbal  communication 
for  most  environments  [20,  21].  Comparing  visual  versus 
verbal/auditory  or  tactile  presentation  of  data,  empirical 
evidence  shows  that  visual  communications  is  more  effective 
[22].  Within  visual  communications,  using  spatial  cues  is 
generally  more  effective  than  using  only  color  cues  [21]. 
Visually  labeling  elements  with  legend  information  in  close 
proximity  minimizes  cognitive  context  switching  but  is  not 
always  observed  in  practice  (the  proximity  compatibility 


TABLE  I  Comparison  of  Challenges  in  Network  Monitoring  Tasks 


Network 

Visualization 

Interactive  Capability 

Proactive 

air  traffic 

radar  displays, 
visual  contact 

voice  communication 

power  grid 

tables,  text, 
maps 

voice  communication, 
physical  inspection 

water 

system 

gauge  indicators 

gathering  data  on-site 

Reactive 

nuclear 

plant 

gauge  indicators 

physical  inspection 

principle)  [23]. 

Operator  decision-making  in  the  network  environments  we 
studied  is  deeply  involved  with  handling  uncertainty.  Often 
operator  must  judge  best/expected/worst  case  outcomes  based 
on  uncertain  inputs  resulting  in  conservative  risk-averse 
decision-making  as  the  standard  codified  in  procedures. 
Counter-intuitive  results  from  [24-26]  report  that 
communicating  uncertainty  to  operators  has  little  or  no  effect 
on  decision-making  -  these  results  may  support  reducing  the 
required  amount  of  information  presented  to  operators  for 
them  to  make  decisions  (eliminating  the  requirement  for 
communicating  uncertainty).  It  is  important,  however,  to 
visually  display  data  resolution  and  uncertainty.  Empirical 
results  also  show  that  representing  levels  of  resolution  in  three 
(or  four)  levels  (such  as  danger,  uncertain,  safe)  as  opposed  to 
just  two  levels  (danger,  safe)  fosters  greater  human  trust  in 
systems  by  reducing  false  negative  and  false  positive  errors 
[27]. 

In  times  of  high  stress,  human  operators  severely  filter 
information  by  limiting  their  focus  to  a  small  subset  of  their 
normal  observation  in  order  to  minimize  cognitive  load.  At 
these  times  of  high  stress  presenting  less  probabilistic 
information  is  preferable  since  the  information  display  can 
induce  specific  behavior.  Much  more  research  is  needed  to 
analyze  human  operator  decision-making  in  handling  stress 
and  processing  uncertain  data. 

V.  Summary 

We  have  surveyed  four  well-established  and  critically 
important  networks  which  are  monitored  by  human  operators. 
Because  generalizing  about  human  monitoring  is  hard  for 
even  a  single  type  of  network,  generalizing  human  monitoring 
across  different  types  of  networks  is  even  more  challenging. 
Despite  this  difficulty,  there  are  a  few  generalizations  that  can 
be  made  from  our  literature  survey:  (1)  visualization  is 
primary  way  to  communicate  complex  network  information  to 
human  operators  and  it  is  either  already  in  use  or  being 
designed  for  use  in  all  of  the  networks  we  studied,  (2)  while 
the  specific  definition  of  situational  awareness  varies  in 
meaning  for  different  network  types,  the  need  to  be  aware  of 
current  status  and  its  relation  to  potential  future  states  was 
found  in  every  network  we  studied,  (3)  human  operator  tasks 
fall  into  two  major  categories  for  the  different  networks  we 
studied  -  proactive  versus  reactive  network  monitoring,  and 
(4)  since  human  operators  have  a  need  to  both  communicate 


with  other  humans  and  query  automation  for  specific 
information,  each  network  we  studied  supported  some  type  of 
interactive  capability  appropriate  for  environment.  Thus  we 
posit  that  the  combination  of  (1)  visualization,  (2)  situational 
awareness,  (3)  proactive/reactive  network  monitoring,  and  (4) 
interactive  capabilities  are  the  four  core  elements  necessary 
for  effective  human  monitoring  of  complex  networks. 
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